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Abstract 

The pseudo-additive relation that the Tsalhs entropy satisfies has 
nothing whatsoever to do with the super-and sub-additivity properties 
of the entropy. The latter properties, like concavity and convexity, are 
couched in geometric inequalities and cannot be reduced to equalities. 
Rather, the pseudo-additivity relation is a functional equation that deter- 
mines the functional forms of the random entropies. The Arimoto entropy 
satisfies a similar pseudo-additive relation and yet it is a first-order ho- 
mogeneous form. Hence, no conclusions can be drawn on the extensive 
nature of the system from either the Tsallis or the Arimoto entropy based 
on the pseudo-additive functional equation. 

Tsallis PP has used the pseudo-additivity condition, 

^^(.4 + B)^ S^{A) + S^{B) + (1 - a)S^{A)S^{B) (1) 
where we work in energy units in which fc = 1, 

So.{P) ^'=} ^' (2) 

a — I 

is the Tsallis entropy V a > 0, and the probability distribution P ~ (pi, . . . ,Pm) 
is complete. Supposedly Q], 

A and B are two independent systems in the sense that the proba- 
bilities oi A + B factorize into those of A and B (i.e., Pij{A + B) = 
Pi{A)pj{B)). We immediately see that, since in all cases Sa > 
{nonnegativity property), a < 1, a = 1 and a > 1 respectively cor- 
respond to superadditivity (superextensivity), additivity (extensivity) 
and subadditivity (subextensivity) . 
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Yet, it appears odd that criteria of super- and sub-additivity can be obtained 
through a functional equation rather than as a geometric inequahty as are the 
criteria for convexity and concavity. The subadditive property is the 'triangle 
inequality' |2], whereas the geometrical interpretation of a concave function is 
one that never rises above its tangent plane at any point. There are classes 
of functions which are defined by inequalities that are weaker than convexity 
(concavity) and stronger than superadditivity (subadditivity) 

To prove that l(2Jl is always subadditive, consider the sum X^I^i Pi ■ Com- 
panions to Minkowski's inequalities are ^ 



E 



fe-fg.r >E p" + E (3) 



for a > 1, and 



for < a < 1, where Q — (gi, . . . , q„i) is another complete distribution. Inequal- 
ity © is the condition for superadditivity; the negative of which is subadditive. 
Hence, for a > 1 the Tsallis entropy is subadditive. Inequality Q is the criterion 
for subadditivity, and, hence, the Tsallis entropy is subadditive for < a < 1. 
This is in flagrant contradiction with the above conclusion that the entropy ((SJ 
is superadditive for < a < 1 on the basis of 

The pseudo-additivity relation |^ is actually expressed in terms of the com- 
plete probability distributions 

So.{PQ) = S^{P) -f S^{Q) + (1 - a)S^{P)Sa.{Q). (5) 

The product of the two sums on the left-hand side does not imply additivity of 
two systems. Rather, Q is a functional equation which determines the form of 
the entropy. Additive entropies of degree- 1 and degree-a can be derived from 
the first- order homogeneous property of the weighted means |4, p. 68] 

m^{\x) = Xm^{x), (6) 

where x stands for a set of discrete variables, xi, . . . , Xm- Since Lp is defined up 
to a constant, we may set 

^(1) = 0. (7) 

Now ^ stands for 

m^{x) = \-^m^{\x) = A-V"' I^E p^vi^^)^ = "^4^)- (8) 

On account of the translational invariancy of the weighted mean 

^{x)^a{\)^{x)+h{\), (9) 
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where a 7^ and b are functions of the parameter A. Since it is apparent from 
((SJ that i/)(a;) = ip{Xx), we get 

ip{Xx) = a{X)ip{x) + b{X) (10) 

However, according to (0 we must set b{X) <p(A). Letting A become another 
positive variable, y, ^ becomes the functional equation 

(p{xy) = a{y)ip{x) + ip{y). (11) 

By the law of associativity 

ip{xy) = a{x)if{y) + if{x). (12) 
Setting the right hand sides of and 112|) equal to one another results in 
a{x) - 1 a{y) - 1 



<p{x) (p{y) 



(13) 



where c is a separation constant. Solving H13|) for a{y) and substituting it into 
Hll() gives the functional equation 

f{xy) = fix) + Lp{y) + cip{x)ip{y) (14) 

which is known in information science as the additivity property of degree-a. 
If c = 0, it reduces to the classical equation 

ip{xy) = ip{x) + (p{y) 

whose most general solution is ip ^ Alogx. 

If X stands for the probability p, then the best code length for an input 
symbol of probability p is 

fiPi) = - log Pi = ^i, (15) 

where A — ~1 and n.i is the length of sequence i. Then the average length, 
Pi'^j corresponds to the average entropy [S] 

m 
i=l 

which is the Shannon entropy, or additive entropy of degree- 1. 

With c ^ 0, we insert cip{x) + 1 = g{x) into H14|) . thereby reducing it to 

g{xy) = g{x)g{y) (17) 

Since the general solution to the functional equation (|17|l is g — x^ , the random 
entropy function is 

fip^) = (18) 
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Since entropies must be positive, we must have c < 0. Moreover, we want lfTH|l 
to reduce to H15() in the Umit as — c +0 so that r — —c. Hence (|18|) becomes 

^ip^) = T- (19) 

a — 1 

where we have set c=l — a<0. In the Umit a | 1 it reduces to ()15|) . The 
average of the entropy ((T^ is the nonadditive entropy of degree-a |Sir7ll5llUl[TUl 
I11[I12[[T^ ^ which reduces to the additive entropy (|16|) in the hmit as a | 1. 

There is nothing unique about H19|) . If our only concern is that it reduce to 
H15|l in the limit a f 1 then an equally likely candidate is 

V{P^) = -T^- (20) 
I — a 

If we average (|2()|l with respect to Q, then we must choose the P in terms of 
Q such that the new P are normalized. In terms of a variational problem we 
require 

1 - Pi 



qi—, '- + Pi 



i=l 



where the constraint has been introduced using the Lagrange multiplier, fi. Per- 
forming the variation we get pi = {qi/fJ-y^", and in order that P be normalized 



l/a 

P. 



1/c 



Em 
^=l 9. 

Introducing these so-called "escort" probabilities back into (|20|l and aver- 
aging give 

^.(o) = ^|i-(E'^f (21) 

where we have set a = 1/ p. The first-order homogeneous entropy (|21|) appears 
to have been first introduced by Arimoto ^B], and subsequently given a complete 
characterization by Boekee and van der Lubbe ■ 

Just as the Tsallis entropy (0) obeys the pseudo-additivity relation the 
Arimoto entropy H21(l satisfies 

SfjiPQ) - SpiP) + SfiiQ) - (^^) Sp{P)Sp{Q). (22) 

Following Tsallis' line of reasoning we would conclude from (|22|) that (|21|l is 
subadditive for (3 > 1 and superadditive for /3 < 1. However, according to 
Minkowski's inequalities 0] p. 31] 
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for /? > 1, and 

for /3 < 1. Thus, j is subadditive for (3 > 1 and superadditive for 

f3 < 1. Since the negative of the former and positive of the latter is used to 
construct the Arimoto entropy, we conclude that it is always superadditive. 

Moreover, concavity follows easily by setting pi = Aa; and = (1 — A)&i, 
where the sets of numbers and bi are nonnegative, and < A < 1. Then 



inequality (|23|l becomes the criterion for the convexity of 



1//3 




^ (Aa. + (1 - A)50M < A [y^ A + (1 - A) ( ^ 5f j 

for /3 > 1, and 

f5^(Aa,: + (l-A)6.)M >a(^ 

for /3 < 1 is the condition for its concavity. Since the Arimoto entropy takes the 
negative of the former and the positive of the latter it is everywhere concave 

m 

Likewise, the concavity of the Tsallis entropy Q can be established from 
the observations that X)"=i -Pf convex for a > 1 and concave for < a < 1. 
Since the Tsallis entropy is formed from the negative of the former and the 
positive of the latter, it is concave for all a > 0. 

Furthermore, since the entropy is defined to within a constant, classically, 
the Arimoto entropy H21|l is a first-order homogeneous function, and, yet, it 
obeys the pseudo-additive relation just like the Tsallis entropy ^ obeys 
the pseudo-additive relation 0. Hence, we can conclude that this relation has 
nothing whatsoever to do with the extensivity properties, or lack thereof, of the 
nonadditive entropies. 
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